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Two Views Part 1: 2D transforms and 
Projective Geometry 


Administrivia 


• PS 2: 

• Get SDD and Normalized Correlation working for a given windows 
size - say 5x5. Then try on a few window sizes. 

• If too slow, resize the images (imresize or take some of the middle) 
and get your code working. Then try on full images (which are 
reduced already!). 

• Results not perfect on even test images? Should they be? 

• Yes you can use normxcorr2 (it did say this!) 

• Some loops are OK. For SSD you might have 3 nested loops (row, 
col, disp) but shouldn’t be looping over pixels. 

• EXPERT: There is a fast way: shift entire right image by d. 
Subtract from left, square, then box filter. This is *all* the numbers 
you need for disparity d for all pixel locations. Do this over all d and 
you have all the data you need to compute best match for every 
pixel. . 
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Administrivia 

• Now: Multiple-views 

• FP 7.1, 8 (all) 


• Today: First half of 2-Views . 


Two Views Part 1: 2D transforms and 
Projective Geometry 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Two views. . .and two lectures 

• Projective transforms from image to image 


• Some more projective geometry 

• Points and lines and planes 

• Two arbitrary views of the same scene 

• Calibrated - “Essential Matrix” 

• Two uncalibrated cameras “Fundamental Matrix” 

• Gives epipolar lines 
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2D Transformations 



Two Views Part 1: 2D transforms and 
Projective Geometry 



X 
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2D Transformations 



Example: translation 

x' — x + t 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


2D Transformations 



Example: translation 

x' — X + t X 1 = 
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2D Transformations 



Example: translation 

x' = x + t 


[ BTW: Now we can chain transformations ] 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Projective Transformations 

• Projective transformations: for 2D images it’s a 3x3 matrix 
applied to homogenous coordinates 
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Two Views Part 1: 2D transforms and 
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Special Projective Transformations 

• Translation 
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• Preserves: 

• Lengths/Areas 

• Angles 

• Orientation 

• Lines 
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Special Projective Transformations 

• Euclidean (Rigid body) 
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• Preserves: 

• Lengths/Areas 

• Angles 

• Lines 
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Special Projective Transformations 

• Similarity (trans, rot, scale) transform 
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• Preserves: 

• Ratios of Areas 

• Angles 

• Lines 
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Special Projective Transformations 

• Affine transform 
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• Preserves: 

• Parallel lines 

• Ratio of Areas 

• Lines 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Projective Transformations 

• Remember, these are homogeneous coordinates 
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Projective Transformations 

• General projective transform (or Homography) 
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• Preserves: 

• Lines 

• Also cross ratios 
(maybe later) 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


The projective plane 

• What is the geometric intuition of using homogenous 
coordinates ? 

• a point in the image is a ray in projective space 



Each point (x,y) on the plane (at z=l) is 
represented by a ray (sx,sy,s) 

- all points on the ray are equivalent: (x, y, 1) = (sx, sy, s) 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Image reprojection 


The irrelevant 
world! 


Basic question 

• How to relate two images from the same camera center? 
• how to map a pixel from projective plane PP1 to PP2 



Answer 

• Cast a ray through each pixel in PP1 

• Draw the pixel where that ray 
intersects PP2 

Observation: 

Rather than thinking of this as a 3D 
reprojection, think of it as a 2D image 
warp from one image to another. 



PP2 



PPl 


Source: Alyosha Efros 
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Projective Geometry 


Application: Simple mosaics 



Image from http://graphics.cs.cmu.edu/courses/15-463/2010_fall/ 
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How to stitch together a panorama (a.k.a. mosaic)? 

• Basic Procedure 

• Take a sequence of images from the same position 

• Rotate the camera about its optical center 

• Compute transformation between second image and first 

• Transform the second image to overlap with the first 

• Blend the two together to create a mosaic 

• (If there are more images, repeat) 

• ... but wait, why should this work at all? 

• What about the 3D geometry of the scene? 

• Why aren’t we using it? 


Source: Steve Seitz 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Image reprojection 



Warning: This model 
only holds for angular 
views up to 180°. 
Beyond that need to 
use sequence that 
"bends the rays" or 
map onto a different 
surface, say, a cylinder. 


mosaic PP 


• The mosaic has a natural interpretation in 3D 

• The images are reprojected onto a common plane 

• The mosaic is formed on this plane 
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Two Views Part 1: 2D transforms and 
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Mosaics 



Obtain a wider angle view by combining multiple images all of 

which are taken from the same camera center. 






CS 4495 Computer Vision - A. Bobick 


Two Views Part 1: 2D transforms and 
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Image reprojection: Homography 


• A projective transform is a mapping between any two PPs 
with the same center of projection 


• rectangle should map to arbitrary quadrilateral 

• parallel lines aren’t 

• but must preserve straight lines 


called Homography 
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Source: Alyosha Efros 
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Two Views Part 1: 2D transforms and 
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Homography 




To compute the homography given pairs of corresponding points in 
the images, we need to set up an equation where the parameters 
of H are the unknowns . . . 
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Two Views Part 1: 2D transforms and 
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Solving for homographies 

P’ = Hp 


wx' 


a 

b 

c 

X 

wy' 

— 

d 

e 

f 

y 

w 


_g 

h 

• 

z 

l 


Can set scale factor i=l. So, there are 8 unknowns. 

Set up a system of linear equations Ah = b 

where vector of unknowns h = [a,b,c,d,e,f,g,h]T 

Need at least 4 points for 8 eqs, but the more the better... 

Solve for h. If overconstrained, solve using least-squares: 

2 


min 


Ah-b 


• Look familiar? (If don't set i to 1 can use SVD) 

• » help mldivide 
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Two Views Part 1: 2D transforms and 
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Homography 



To apply a given homography H 

• Compute p’ = Hp (regular matrix multiply) 

• Convert p’ from homogeneous to image 
coordinates 
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Mosaics 


Two Views Part 1: 2D transforms and 
Projective Geometry 



Combine images with the computed homographies. . . 


image from S. Seitz 






CS 4495 Computer Vision - A. Bobick 


Two Views Part 1: 2D transforms and 
Projective Geometry 


Mosaics for Video Coding 


• Convert masked images into a background sprite for 
content-based coding 







CS 4495 Computer Vision - A. Bobick 
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Homographies and 3D planes 


• Remember this: 
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• Suppose the 3D points are on a plane: 

aX + bY + cZ + d — 0 
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Two Views Part 1: 2D transforms and 
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Homographies and 3D planes 


• On the plane [abed] can replace Z: 
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• So, can put the Z coefficients into the others: 
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3x3 Homography! 
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Image 


• Mapping between planes is a homography. Whether a 
plane in the world to the image or between image 
planes. 
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Two Views Part 1: 2D transforms and 
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What else: Rectifying Slanted Views of 
Planes 
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Rectifying slanted views 


Two Views Part 1: 2D transforms and 
Projective Geometry 



Corrected image (front-to-parallel) 
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Two Views Part 1: 2D transforms and 
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Measuring distances 
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Two Views Part 1: 2D transforms and 
Projective Geometry 



Measurements on planes 


Approach: unwarp then measure 

What kind of warp is this? 
Homography. . . 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Image rectification 



A planar rectangular grid in the scene. Map it into a 
rectangular grid in the image. 
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Some other images of rectangular grids. . . 


0 
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Who needs a blimp? 


0 


Two Views Part 1: 2D transforms and 
Projective Geometry 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Same pixels - via a homography 





CS 4495 C 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Image warping 



Given a coordinate transform and a source image 
f(x,y), how do we compute a transformed image 
g(x\y) = f(T(x,y))7 


Slide from Alyosha Efros, CMU 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Forward warping 



• Send each pixel f(x,y ) to its corresponding location 

• ( x’,y ’) = T(x,y) in the second image 

Q: what if pixel lands “between” two pixels? 


Slide from Alyosha Efros, CMU 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Forward warping 



g(x’,y’) 


• Send each pixel f(x,y ) to its corresponding location 

• ( x’,y ’) = T(x,y) in the second image 

Q: what if pixel lands “between” two pixels? 

A: distribute color among neighboring pixels (x’,y’) 

- Known as “splatting” 

Slide from Alyosha Efros, CMU 
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Two Views Part 1: 2D transforms and 
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Inverse warping 



Get each pixel g(x’,y’) from its corresponding location 
(x,y) = T 1 (x’,y’) in the first image 

Q: what if pixel comes from “between” two pixels? 


Slide from Alyosha Efros, CMU 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Inverse warping 



Get each pixel g(x’,y’) from its corresponding location 
(x,y) = r I (x’,y’) in the first image 


Q: what if pixel comes from “between” two pixels? 
A: Interpolate color value from neighbors 


- nearest neighbor, bilinear... 

Slide from Alyosha Efros, CMU 


» help interp2 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


Bilinear interpolation 

Sampling at f(x,y): 
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Slide from Alyosha Efros, CMU 
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Recap: How to stitch together a 
panorama (a.k.a. mosaic)? 

• Basic Procedure 

• Take a sequence of images from the same position 

• Rotate the camera about its optical center 

• Compute transformation (homography) between second 
image and first using corresponding points. 

• Transform the second image to overlap with the first. 

• Blend the two together to create a mosaic. 

• (If there are more images, repeat) 


Source: Steve Seitz 
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Two Views Part 1: 2D transforms and 
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Why projective geometry? 
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Two Views Part 1: 2D transforms and 
Projective Geometry 


The projective plane 

• What is the geometric intuition of using homogenous 
coordinates ? 

• a point in the image is a ray in projective space 



Each point (x,y) on the plane is represented by a 
ray (sx,sy,s) 

- all points on the ray are equivalent: (x, y, 1) = (sx, sy, s) 
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Homogeneous coordinates 

2D Points: 
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Projective lines 

• What does a line in the image correspond to in 
projective space? 



A line is a plane of rays through origin 
- all rays (x,y,z) satisfying: ax + by + cz = 0 


in vector notation : 0 = [a b cj 
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A line is also represented as a homogeneous 3-vector I 
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Point and line duality 

• A line 1 is a homogeneous 3-vector 

• It is 1 to every point (ray) p on the line: l r p=0 



What is the line I spanned by rays Pi and p 2 ? 

• I is 1 to p x and p 2 => I = Pi x p 2 

• I is the plane normal 

What is the intersection of two lines \ 1 and l 2 ? 

• p is 1 to l x and l 2 => p = li x l 2 

Points and lines are dual in projective space 

• given any formula, can switch the meanings of points and 
lines to get another formula 
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Homogeneous coordinates 

Line joining two points: 



ax + by + c = 0 


Pi = [*i yi 1] 

p 2 = [x 2 y 2 1] 


/ = Pl X p 2 
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Homogeneous coordinates 

Intersection between two lines: 



k = [°i b i c i] 
l 2 = [a 2 b 2 c 2 ] 
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3D projective geometry 

• These concepts generalize naturally to 3D 

• Recall the equation of a plane: 

aX + bY + cZ + d — 0 

• Homogeneous coordinates 

• Projective 3D points have four coords: p = (wX,wY,wZ,w) 

• Duality 

• A plane N is also represented by a 4-vector N =(a,b,c,d) 

• Points and planes are dual in 3D: N T p = 0 

• Projective transformations 

• Represented by 4x4 matrices T: P' = TP 
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3D to 2D: “perspective” projection 


• Matrix Projection: 
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You’ve already seen this. 

What is not preserved under perspective projection? 


What IS preserved? 
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What’s next. . . 

• Today - more projective geometry, the duality between 
points and lines in projective space 

• Tuesday- using the projective geometry revisit 2-views: 
• Essential and Fundamental matrices 



